Automatically embedding newsworthy links to articles: From implementation to evaluation

News portals are a popular destination for web users who read news online. News providers are therefore greatly interested in attaining higher visitor rates and promoting greater engagement with their content. One aspect of user engagement deals with keeping users on site longer, by allowing them to navigate through enhanced click-through experiences. Therefore, news portals have invested in ways to include embedded links within news stories. So far these links have been curated by news editors, who analyse the content and identify newsworthy events, suitable for linking to archived relevant articles. However, due to the manual effort, the use of such links is limited to small-scale. In this paper, we describe and evaluate a system-based approach that detects newsworthy events in a news article and locates other articles related to these events. Our system possesses two important characteristics. Firstly, it does not rely on resources like Wikipedia to identify events, since newsworthy events will often not be contained in them. Secondly, it was designed to be domain independent. A rigorous evaluation, harnessing the crowd sourcing power of Amazon’s Mechanical Turk, was performed to assess the system-embedded links against the manually-curated and understand the resulting reading experience. Our findings reveal that our system’s performance is comparable to that of professional editors, and that users find the automatically generated highlights interesting and the associated articles worthy of reading. Our evaluation also provides quantitative and qualitative findings that helped understand better the curation of links from the perspective of users and professional editors.